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DETAILED ACTION 

Response to Amendment 

1. Applicant's amendment filed on October 21, 2008 has been considered and entered in full. 

2. Applicant's amendments to the claims and arguments with respect to amended claims have 
been considered but are moot in view of the new ground(s) of rejection(s) made below. Claims 1-8 
and 17-19 are subject to new rejections under 35 USC 101 due to the recent judicial changes and 
that is why this office action is a non-final action. Based on the applicant's amendments to the 
claims, 35 USC 112 rejections on the respective claims have been withdrawn. 

Claim Rejections - 35 USC § 101 

3. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any 
new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of 
this title. 

The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Gazette notice of 22 November 2005 
<http://www.uspto.gov/web/offices/com/sol/og/2005/week47/og200547.htm>), Annex 
IV, reads as follows: 

In contrast, a claimed computer-readable medium encoded with a computer program is a 
computer element which defines structural and functional interrelationships between the 
computer program and the rest of the computer which permit the computer program's 
functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 
1035. 
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Claims that recite nothing but the physical characteristics of a form of energy, such as a 
frequency, voltage, or the strength of a magnetic field, define energy or magnetism, per se, and 
as such are nonstatutory natural phenomena. O'Reilly, 56 U.S. (15 How.) at 112-14. Moreover, it 
does not appear that a claim reciting a signal encoded with functional descriptive material falls 
within any of the categories of patentable subject matter set forth in Sec. 101. 

... a signal does not fall within one of the four statutory classes of Sec. 101. 

. . . signal claims are ineligible for patent protection because they do not fall within any of the 

four statutory classes of Sec. 101. 

4. Claims 12-16 and 21-22 are rejected under 35 U.S.C. 101 because: 

Claims 12-16 and 21-22 are drawn towards a computer-readable medium having a computer 
program or instructions embodied thereon. Such computer executable programs are functional 
descriptive material that can constitute statutory products when properly claimed in combination 
with a computer readable storage device that allows the functionality of the program to be realized. 
However, while the claim defines the program in combination with a "computer usable medium," 
the claim language is not limited to the use of tangible storage media in combination with the 
program. Specifically, the disclosure defines the computer accessible/ readable medium as including 
read only memory (ROM); random access memory (RAM); magnetic disk storage media; optical 
storage media; flash memory devices; electrical, optical, acoustical or other form of propagated 
signals (e.g., carrier waves, infrared signals, digital signals) (paragraph 0020 on page 9 of the 
specification). While the read only memory (ROM); random access memory (RAM); magnetic disk 
storage media; optical storage media; flash memory devices (statutory media) are physical storage 
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devices, but the carrier wave/infrared signal/digital signal is a signal which is intangible energy, and 
therefore is not considered to be a statutory product. While the claim does read on at least some 
statutory embodiments (such as the floppy or hard or magnetic or optically readable disks), it also 
encompasses the combination of the program with the carrier wave, which is not a statutory 
product. Since the claimed invention encompasses non-statutory embodiments, the claim is directed 
towards non-statutory subject matter. 

Claims 12-16 and 21-22 as a whole define a carrier wave, and "[a] transitory, propagating signal 
...is not a "process, machine, manufacture, or composition of matter." Those four categories define 
the explicit scope and reach of scope and reach of subject matter patentable under 35 U.S.C § 101; 
thus, such a signal cannot be patentable subject matter." (In re Petms A.C.M. Nutijen; fed Cir, 2006- 
1371, 9/20/2007). 

Because the full scope of the claim as properly read in light of the disclosure encompasses 
non-statutory subject matter, the claim as a whole is non-statutory. The examiner suggests amending 
the claim to include the disclosed tangible computer readable media such as "computer readable 
storage medium", while at the same time excluding the intangible media such as signals, carrier waves, 
etc. Any amendment to the claim should be commensurate with its corresponding disclosure. 

5. Claims 1-8 and 17-19 are rejected under 35 U.S.C. 101 as not falling within one of the four 
statutory categories of invention. Supreme Court precedent {Diamond v. Diehr, 450 U.S. 175, 184 
(1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk v. Benson, 409 U.S. 63, 70 (1972); 
Cochrane r. Dealer, 94 U.S. 780, 787-88 (1876)) and recent Federal Circuit decisions {In re Bilski, 88 
USPQ2d 1385 (Fed. Cir. 2008)) indicate that a statutory "process" under 35 U.S.C. 101 must (1) be 
tied to another statutory category (such as a particular apparatus), or (2) transform underlying 
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subject matter (such as an article or material) to a different state or thing. While the instant claim (s) 
recite a series of steps or acts to be performed, the claim(s) neither transform underlying subject 
matter nor positively tie to another statutory category that accomplishes the claimed method steps, 
and therefore do not qualify as a statutory process. For example, the method including steps of 
receiving and dynamically acting on a characteristic data, is of sufficient breadth that it would be 
reasonably interpreted as a series of steps completely performed mentally, verbally or without a 
machine. The applicant has provided no explicit and deliberate definitions of "receiving" and 
"dynamically acting on a characteristic data" to limit the steps to electronic processing. There is no 
evidence in the claims of what receives the eyetracking and external context data and what performs 
the step of "dynamically acting on a characteristic data". Examiner suggests amending the claims by 
inserting a statutory category in the body of the claims so that the method steps are positively tied to 
a statutory category. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent ma\ not he obtained though ilic invention is in it idem icalh disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 

7. Claims 1-2, 4-5, 7-10, 12-13 and 15-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hatfield et al., November 30, 1997, "An Interface integrating eye gaze and voice 
recognition for hands-free computer access" (pages 1-7), further in of view of Newman et al., U.S. 
Patent No. 5,844,824. 
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Hatfield discloses an EVA (Eye/Voice Aware) interface which allows combined eye-tracking 
and voice input to allow hands-free interaction with a standard window-based computer application, 
where voice input being the external context data. 

Hatfield discloses "In EVA, we implemented a mode of operation where the user can slave the 
screen cursor to his/her eye. Wherever the user looks on the display while operating in this mode, 

the cursor will be displayed. . .The user fixates a Windows button The system will detect rapid 

eye movement, e.g., saccades and adjust very quickly to gross changes in POG "(Hatfield, topic 4 - 
Initial EVA Implementation, pages 4 and 5). Clearly from Hatfield's disclosure, it is clear that the 
cursor is a slave of the eye, and the cursor motion is defined by the eye movement (or eye tracking 
data) and it being apparent that in order to interpret the eye motion (or eye tracking data), an eye 
interpretation engine (or system or component) is inherendy required which would send the 
translated data to the system to synchronize the cursor with the eye motion (or eye behavior), 
without which EVA system won't be able to synchronize the eye with cursor. The interpretation 
component is a must part of a system which uses eye motion or eye behavior to control a system. 
Thus, EVA system, receiving from an eye interpretation engine, at least an interpretation of 
eye tracking data corresponding to at least one user. 

Hatfield further discloses receiving external context data corresponding the at least one 
user's interaction with an application (Topic 4 - Initial EVA Implementation- pages 4 and 5 - 
external context data being voice commands such as "forward tab", "click", "page up" and "page 
down" — With IE users can navigate the various hyperlinks in a web page by "tabbing between 
them" with the command "forward tab". They can also fixate a particular hyperlink and utter 
"click", page 4 -last few lines). Now, as in Microsoft Windows, the operations of the computing 
device running the application are controlled by the use of cursor and the buttons, and controlling 
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the system is basically controlling the operational state of the processor of the computing device. 
Hatfield discloses that voice input is used as command entry for controlling the operational states of 
the processor of the computer running the application. For example, voice commands such as 
"forward tab, "page up" and "page down" would control the operational state of browsing of 
internet explorer in the device (pages 4, last few lines through page 5), which conforms to the 
limitation "wherein the external context data is related to an operational state associated with a 
computing device running the application". 

Hatfield further discloses "based, at least in part, on the interpretation of the eye tracking 
data and the external context data, dynamically acting on a characteristic of the application" (Topic 4 
- Initial EVA Implementation - page 4. last few lines through page 5 - in internet explorer, fixate a 
particular hyperlink and utter "click" which has the effect of clicking with a mouse on the hyperlink, 
thus hyperlink being the characteristic of the application (internet explorer) and dynamically acting 
on the characteristic being clicking on the hyperlink to invoke the internet explorer to display the 
contents of the link on the browser). 

As discussed before, Hatfield discloses an EVA (Eye/Voice Aware) interface which allows 
combined eye-tracking and voice input to allow hands-free interaction with a standard window- 
based computer application, where voice input being the external context data. Hatfield provides an 
example of an application in which the combination of eyetracking and voice commands is used by a 
single user of the computer but does not expressly teach the usage of the combination of 
eyetracking and voice by a plurality of users. 

However, Newman discloses a handfree system in which the user may operate the 
computing apparatus to display information in a hands-free manner utilizing eye tracking with or 
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without audio commands (col. 2, lines 61-64). Newman further discloses "the system concept of 
eye-tracking permits the operator to manipulate the computer completely hands-free by tracking the 
eye and interpreting its movements as "mouse" commands to the application. Generally, the eye 
tracking system is head-mounted on the user but may be used in any suitable manner" (col. 3, lines 
40-45). Newman further discloses "a voice command can be given to the present computer which is 
confirmed or enhanced by the present eye-tracking system and vice-versa" (col. 3, lines 42-45). 
Newman further discloses "the system can work in two modes: In the first mode, a user having 
a headset with activation means, display screen and communication means is linked with a host 
computer either worn on the user's person contained within the display means or remote from the 
person. The user would command the host computer to display a particular procedure for repairing 
a piece of equipment. The host computer would then search for an transmit the procedure back to 
the user for display on the user's headset. In the second mode, a host computer could also be 
responsive to several users having headsets that may be working as a team to repair the 
piece of equipment. The users could all work form the same procedure or call for a different 
procedure to be displayed as an additional window in the display' (col. 7, lines 1-15). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time invention was 
made to combine the teachings of Newman of extending the combination of eye tracking and voice 
data to multiple-users to control/ operate the computer with that of Hatfield. One of ordinary skill in 
the art would have been motivated to combine the teachings of Newman with that of Hatfield 
because both references are directed to the same field of endeavor and Newman's teachings when 
combined with the teachings of Hatfield would enhance or provide the Hatfield's invention with 
additional capability or versatility of handling applications where multiple users concurrently use the 
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computer to share the same application when working on the same task, and would further increase 
the efficiency of the group or users working on the task. 

Regarding claim 2, claim 2 recites wherein acting on comprises one of: modifying, creating, 
destroying, removing, invoking and configuring. Hatfield discloses "in internet explorer, fixate a 
particular hyperlink and utter "click" which has the effect of clicking with a mouse on the 
hyperlink", (Topic 4 - Initial EVA Implementation - page 4. last few lines through page 5), thus 
hyperlink being the characteristic of the application (internet explorer) and dynamically acting on the 
characteristic being clicking on the hyperlink to invoke the internet explorer to display the contents 
of the link on the browser. 

Regarding claim 4, all the limitations in the claim 4 are similar to claim 1 except that claim 4 
recites "dynamically acting on an output of the application based, at least in part, on the 
interpretation of the eye tracking data and the external context data". As discussed before, Hatfield 
discloses (Topic 4 - Initial EVA Implementation - page 4. last few lines through page 5 - in internet 
explorer, fixate a particular hyperlink and utter "click" which has the effect of clicking with a mouse 
on the hyperlink) thus hyperlink being the characteristic of the application (internet explorer) and 
dynamically acting on the characteristic being clicking on the hyperlink to invoke the internet 
explorer to display the contents of the link on the browser. Thus clicking on the hyperlink would 
invoke the internet explorer to display the contents of the link as output on the browser, where the 
output again being a page in the internet explorer with hyperlink. Hatfield further discloses acting on 
the output (page 5, topic 4, 1 st few lines - Users can also scroll the browser by issuing "page up" and 
"page down" commands, open list boxes and select various items). 
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Claim 5 has been similarly analyzed and rejected as per claim 2. 

Claims 7-10, 12-13 and 15-16 have been similarly analyzed and rejected as per claims 1-2 and 

4-5. 

Regarding claims 17-22, Hatfield discloses the external sources (external context data) 
comprise one or more of: system information, location and/or z-order of windows and/or objects, 
document Object Model (DOM) of a Web page or application being viewed, current application 
process state and/or visual state, task models, cognitive models describing the mental or physical 
steps or states required (page 4 - last few lines - current application process state and/ or visual 
state - tabbing between the hyperlinks which being the process/visual state of browsing, the tasks 
model including tabbing and scrolling, fixating and clicking a windows button). 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent nv,i\ noi he obtained though the invention is not identically disclosed or described as set forth in section 
102 of this tide, if the differences between the subject matter sought to be patented and the prior an are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived b\ the manner in which the 

9. Claims 3, 6, 11 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hatfield et al., November 30, 1997, "An Interface integrating eye gaze and voice recognition for 
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hands-free computer access" (pages 1-7), further in of view of Newman et al., U.S. Patent No. 
5,844,824, and further in view of Martins, U.S. Patent No. 6,608,615 Bl. 

Regarding claims 3, 6, 11 and 14, as discussed in the rejection of claims 1 and 4, the 
combined invention of Hatfield and Newman discloses that eye tracking data can be used to control 
various aspects of the interface for example controlling the browsing in internet explorer (IE) and 
selecting the various items in IE but combined invention of Hatfield and Newman does not 
explicitly disclose the extended applications where the eye tracking data would be used such as 
"determining from the interpretation of the eye tracking data at least a portion of an interface that 
has not been viewed by the user and modifying a format of the portion of the interface that has not 
been viewed by the user". However, Martins which belong to the same field of endeavor describes 
the extended applications where the eye tracking data can be used. Martins discloses "if the banner 
add is not viewed at all, the intervention would cause the banner to dynamically appear to jump 
closer to the user's gaze" (Martins, col. 6, lines 19-21). Therefore, it would have been obvious for 
one of ordinary skill in the art at the time of invention was made to combine the teachings of 
Martins with that of combined invention of Hatfield and Newman. One of ordinary skill in the art 
would have been motivated to combine the teachings of Martins with that of combined invention of 
Hatfield and Newman as it would enhance the capabilities of combined invention of Hatfield and 
Newman to be used in extended appUcations, thus making Hatfield's system more versatile in the 
field of controlUng the interface application using eye tracking data. 

Examiner note: Examiner has cited particular columns and line numbers in the references as 
applied to the claims above for the convenience of the applicant. Although the specified citations are 
representative of the teachings for the art and are applied to the specific limitations within the 
individual claim, other passages and figures may be applied as well. It is respectfully requested from 
the applicant in preparing responses, to fully consider the references entirely as potential teachings 
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all or part of the claimed invention, as well as the context of the passage as taught by the prior art or 
disclosed by the examiner. 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can 
normally be reached on Monday to Friday from 8:30 am to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Matt Bella, can be reached on (571) 272-7778. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR system, 
see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, 
contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



/Manav Seth/ 
Examiner, Art Unit 2624 
January 13, 2009 



